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ABSTRACT 

As a result of the increasirg insistence that schools 
should concentrate on teaching fcasic skills, states and school 
districts have moved to establish minittUE ccitpetencies for high 
school graduation, to be assessed by tests that demonstrate that 
"students have achieved a certain level , cf^ 2a~ste"ry^ c£ £asic' skTlTs, 
The Austin, Texas, Independent School District' s beard of trustees 
passed in 1975 the first version of the minimuE competency testing 
requirement for Austin public school students. It stated that 
students not demonstrating an eighth grade competence (as measured by 
standard achievement tests) in reading and nath cannot graduate from 
Austin high schools. The problems encountered in implementing the 
board's mandate, such as test selection, test security, and test 
administration, are described in this speech, (fiuthor/DS) 
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The cry has been growing for some time — students graduate from high 
school and arrive at college or a job without even minimum adequacy in very 
basic subjects. They can't read instructions. They can't balance their 
checkbooks. They can't even write a coherent sentence. The meaning of 
a high school diploma has been diminishing for some time. As early as 
1958, a survey carried out by the city of Denver showed that most business- 
men considered a high school diploma to indicate attendance in school for 
12 years, but not much more.^ Similar opinions have been expressed in a 
variety of other forums. In addition, a survey carried out by the University 
of Texas showed that between 1/5 and 1/3 of adults have difficulty func- 
tioning in such basic areas as math computations, health, and consumer 
economics.^ The public, faced with these findings, declining SAT scores, 
and many other similar bits of information, is increasingly demanding to 
Imow why today *s high school education does not in fact seem to educate. 

Some of the answers are clear. Today's schools have too many tasks. 
They carry the major burden of integrating society. They teach students 
to like and value themselves and others. They try to provide each student 
an education which allows him to explore who he is and where he is going. 
Perhaps, with so many goals competing for instructional time, it's not too 
surprising that not everyone learns to read. 

In response to such competing priorities there has grown up the '*back 
to basics'' movement — an insistence that schools should concentrate first 
of all on teaching some very basic skills like reading. And following in 
the wake of this movement has grown another — the trend toward minimum 
high schools graduation competencies, assessed by tests which demonstrate 
that students have indeed learned these basics. 
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Standardized testing at the high school level Is not new, of course. 
Achievement tests, aptitude tests and college entry tests have been given 
In high schools for many years. However, along with the movement toward 
more ^'humanized" education of the 1960s and early 1970s came a lessened 
emphasis' on testing. In addition, controversy over the use of aptitude 
and IQ tests for minority students for tracking and placement purposes 
grew, and the general issue of test bias began to receive a great deal of 
attention. Although this controversy was focused primarily on aptitude 
tests, some negative feelings inevitably spilled over onto standardized 
tests of all kinds. Thus testing of any kind was, for a time, given re- 
latively low priority. 

Many concerns over testing and the use of tests remain. Educators 
express fears that tests used for graduation requirements will dictate a 
very narrow curriculum specifically aimed at the test. They feel tnat 
a m inimum competency will also tend to become a maximum goal, beyond 
which students won't try to learn and teachers won't try to teach. ^ 
They are concerned about the "labeling^ effect of yet another test, and 
the consequences of telling substantial numbers of teenagers that they are 
not competent.^ They are concerned that the testing will force substantial 
changes in the entire school system, ranging from the kinds of courses 
taught to the way progress is reported to parents, and especially con- 
cerned that these changes are not foreseen and planned for.^ They worry 
tnat resources for adequate test development and administration will not 
be made available, leading to the use of restrictive tests developed in 
isolation from a district's curriculum.^ 
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Nonetheless > In spite of these concerns) competency testing Is clearly 
on the way. In 1969 Denver initiated its Proficiency and Review Test 
program, which still continues. Three years later, in 1972, Oregon became 
the first state to mandate a minimum competency test for high school grad- 
uation. 

In 1975, Florida and California passed minimum competency test bills, 
and a number of individual school districts, including Gary, Indiana; 
Anchorage, Alaska and Austin, Texas were involved in setting up tests for 
graduation. By May cf 1976, there were 7 states with legislation or 
education board rulings mandating competency testing of some kind, and 
6 more with legislation pending. Individual school districts also con- 
tinued to set such programs. By January of 1977, 17 states had set some 

8 9 

kind of minimum competency program. * 

At that point the movement really escalated. In a September, 1977 
publication, ETS listeci 20 states as experimenting with or piloting^ some 
kind of competency test program, 16 more as planning one, and 11 more 
as studying the matter. Only three states had no plans for minimum 
competency testing activities. 

Many of these minimum competency tests at the state level are de- 
signed primarily for assessing student progress throughout school, rather 
than for graduation requirements. But by December, 1977 there were 18 
states which required minimum competency tests for graduation from high 
school of all students, and 12 more which required minimum competency 
tests, but did not specify for high school graduation. Of the 18 re- 
quiring minimum competency testing for graduation, two are requiring it 
already, 7 more have set a date by which time it will be requirf»d which 



Page 4 

is later than June 1978, and 9 are working on a test or otherwise 

12 

studying the procedure. In addition to the activity at the state level, 
many local school districts are now involved in some kind of competency 
testing. For instance, of the seven largest school districts in Texas, 
only one is not testing, considering testing or studying the idea of 
testing its seniors before graduation. In addition, discussion is under- 
way regarding a federal bill requiring minimum competency testing nationwide. 

Typically, euch requirements are in basic academic subjects such as 

rea^ \g, writing, and math, or in applications of these subjects to life 

skills such as filling out forms, calculating income taxes, etc. Some 
requirements extend beyond such basic applications into competence in a 
variety of adult activities* For instance, items included might 
measure not only ability to balance a budget, but also general knowledge 
about how to set one up, wise use of credit, and other aspects of finan- 
cial competence. Most such tests are paper and pencil, and most systems 
provide for initial administration of the test long before twelfth grade, 
to allow time for remediation. 

Beyond this general summary of minimum competency testing, however, 
there are a variety of detailed questions which must be faced by any 
school district about to implement a program such as this. While passing 
the initial legislation or policy is relatively easy, going from the 
mandate to actual test scores is another matter entirely. How one district 
dealt with these questions, and problems that were encountered is dis- 
cussed below. 

Wlien a school district sets minimum competency requirements for grad- 
uation, some of the implications are quite clear. For example, it becomes 
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apparent that arrangements must be made for personnel to administer 
and monitor the tests and, in some cases, to score test results; adminis- 
tration schedules mubt be set with adequate informational materials being 
prepared for those administrators, counselors, and other persons who will 
be involved in the administration of the tes^s, as well as students; 
suitable times and places must be set aside for testing; also, parents, 
teachers, and students must be informed of the requirements and of when 
and how the requirements are to be met. 

Other implications may be revealed in the wording of the policy 
statement regarding the proficiency requirements. For example, the 
statement will probably identify the subject areas to be tested (e.g., 
math and/or reading, etc.) and the level of proficiency to be demonstrated 
(e.g., eight grade level). This implies that 8th grade tests in math and 
reading are intended for the competency testing. 

Still other Implications may exist but are likely to be more subtle 
sucn as: Are new students (transfers) to the district subject to the 
same requirements even though they have already completed much of their 
high school work elsewhere? What -ibout students in special education 
programs? Are they subject to the requirements? How are students to 
be identified as having passed or failed the proficiency exams? ,Will 
all students recaive the same diploma or certificate of graduation even 
though some passed the requirements and others were exempt from having 
to meet them. 

In any case, those charged with the implementation and management of 
competency testing program as likely to spend many an hour working on 
expected and unexpected problems that arise. 
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I will share with you some of the problems experienced by the Austin 
Independent School District In planning and Implementing Its competency 
testing program. The first problem that had to be considered was the 
policy statement Itself. In 1975, the first version of the minimum 
competency testing requirement was passed by the Austin Independent School 
District (AISD) Board of Trustees. It stated: 

"Students not demonstrating at least an 8th grade reading level, 
as measured by the California Achievement Test, will be placed In 
classes to remedy Indicated deficiencies. Including courses beyond 
the 9 quarters of required Language Arts unless the student places 
on file a letter signed by a parent or guardian acknowledging that 
the student proposes to graduate without achieving an 8th grade 
reading level.*' 

'*A11 students will begin their required Mathematics in the fall 
quarter 9th grade and continue with Mathematics courses each quarter 
until their Mathematics requirement is met. Students not success- 
fully completing 3 quarters of Algebra or its equivalent will be 
given a Mathematics test covering the 48 basic competencies as de- 
fined by the National Council of Teachers of Mathematics and that 
for students who fail to achieve a 75% performance score, further 
courses will be required to remedy the deficiency unless the student 
places on file a letter signed by parent or guardian acknowledging 
that the student proposes to graduate without basic skills in Mathe- 
matics as specified by the NCTM/'*^^ 

The Director of the Office of Research and Evaluation points out 
that test management problems were inherent in the working of tue Austin 
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policy statement. Several problems surfaced. First of all, the 
statement named a specific test, the California Achievement Test (CAT), 
which had recently been replaced earlier for general testing in the Auctin 
high schools because of its limitation in measuring achievement at the 
high school level page* Instead, the Sequential Tests of Educational 
Progress (STEP Tests) had been selected for the districtwide testing 
program. 

How were the differences to be resolved? Were scudents to be re- 
quired to take both tests? If the high school student were to take the 
8th grade CAT, how was the score to be interpreted? Other problems re- 
sulted from the wording ot the statement. There was the ambiguity regard- 
ing the exemption letter and remedial classes. At what point could the 
student choose the letter option and avoid further remedial classes? 
Also, there was the question of whether students who had met the AISD 
competency requirement must still meet the TEA mandated course require- 
tnents in mathematics and language arts. 

After hours and months of deliberation between board and staff 
members, the requirement was revised in December, 1976. The approved 
revision read as follows: 

"Students will be required to demonstrate at least an eighth grade 
competence in reading or place on file a letter signed by parent or 
guardian acknowledging that the student proposes to graduate without 
achieving an eighth grade level in reading. 

"Students will be required to demonstrate at least an eighth grade 
competence in mathematics or place on file a letter signed by parent 
or guardian acknowledging that the student proposes to graduate 
without achieving an eighth grade level in mathematics.'' 
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This revision was accompanied by '^n administrative procedure state- 
ment which set forth the details for how the requirement could be met* 
Tlie procedure statement read: 

The Austin Independent School District graduation requirement of 
an eigth grade competence in reading and mathematics may be met by any 
of the following options: 

1) Achieving a score on the eighth grade administration of the 
California Achievement Test which represents national median 
performance for eighth grade students • 

2) Achieving the raw score equivalent to the national median per- 
formance by eighth grade students on any annual administration 
of the Sequential Tests of Educational Progress • 

3) Achieving a score on the California Achievement test (level 4) , 
at any re-administration subsequent to high school entry, which 
represents national median performance for eighth grade students. 

Only after the revision had been made could the Department of Secondary 
Education and the Office of Research and Evaluation attend to the specific 
details of Implementation. Within the Department of Secondary Education 
numerous decision questions had to be addressed: 

1) When could/should students bring letters? Were students to 
complete the other graduation required coursework in math and 
language arts and tlien bring the letter: Could they avoid com- 
pleting the course requirements? 

2) What about transfer students? Should all transfer students be 
subject to the competency requirements? What about those students 
who had already earned some of their senior credits at the time 

of the transfer? 



ERIC 




Page 9 

3) What about special education students? Are they to be required 
to meet the competency requirements? Should special waivers or 

« 

exemptions be granted to some of the special education students? 

Some of these questions were addressed immediately as they were pre- 
requisites to implementation. Other questions and their solutions have 
been evolving, more or less. All of the issues mentioned, however, pre- 
ceeded the administrative procedural problems that still had to be addressed 
by the Office of Research and Evaluation, the office having primary 
responsibility for admiiiistering the competency tests. 

The requirement would be effective vith the group of students who 
would be seniors in 1978-79. They were the 10th graders in 1976-77. 
Those students in 10th grade were now nearing the end of their course 
requirements in mathematics. Many of them had taken the CAT in 8th 
grade and/or the STEP in 9th grade prior to the new policy requirements. 
Then had no idea at the time that they took the tests that graduation 
depended on the student's passing of the test. Many had not passed the 
tests. It was decided that these students should be afforded another 
opportunity to meet the requirements before being placed in remedial 
classes. Consequently the first administration of the competency tests 
in the district would be geared to this group of students. Some of the 
quest ions /problems which had to be addressed by ORE before this effort 
took place were: 

1) VThich students must be tested? How would these students be 
identified and informed? 

2) What additional personnel would be needed to help identify 
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students needing to be tested? Have such personnel already had 
their TB tests? Where would funds for hiring and training them 
come from? Would additional persons need to be hired to help 
administer and monitor the test session? How much training would 
be needed? Who woula score the test? etc. etc. 

3) 'Jhat form of the test would be used? What additional materials 
must be purchased? What materials needed to be ordered? What 
was already on hand? 

4) How would the data be recorded, stored and retrieved? How much 
programming time would be needed? \fliat about the computer pro- 
gram to be used? Wno pays for dzcs, programming costs? 

5) How would results be reported to schools, students, and to 
parents? 

The Office of Research and Evaluation developed a timeline that con- 
sidered each of these taoks that were implied. Counselors were contacted 
to assist in identifyng .uaents at 13th grade who had not yet passed 
the graduation requirements, tezt answer sheets vere ordered; a quadru- 
plicate reporting form for test scores was desig led and ordered; test 
sites were scheduled and set up; additional personnel Wi.re hired to ad- 
minister, monitor and score the test. New personnel were trained and 
the testing began. Tlie quadruplicate forms that were designed for record- 
ing the scores allowed the ORE to have the results hand scored immediately 
after the testing sessions and to retain a copy of the scores for the 
records to be checked against the machine scored results that would be 
done later. It also provided three copies of the hand-sccred results to 
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the schools within a twenty-four hour period — thus allowing sufficient 
time to inform students of whether they needed to register for Fundamentals 
of Math Tutorial classes (FOMT) (the ronedial classes) prior to the 
registration period for spring (fall registration). Tne testers were paid 
and forms were disseminated to the schools indicating test results. 
Finally the first round of competency testing had been completed. New 
problems began to evolve iimnediately . What about test security for future 
testing? Since the tests must be given each quarter how was the office 
to ensure that test items did not become available to students before- 
hand? A different form of the test must be ordered before the fall testing. 

Will the final exam for the FOMT classes be administered by ORE or 
by the teacher? Since these students must meet the proficiency require- 
ments it was logical that the final exam would be a form of the standard- 
ized test (CAT). Would it be the same form? If so, again thote are the 
concerns over test security. 

Wliat about data processing of the test result? Can we have all of 
the programming bugs worked out in time to get the needed information 
to counselors, registrars, teachers « and principal before beginning to 
schedule the next round of tests? Have all special students been iden- 
tified? What about bad (erroneous) student ID numbers? 

And then there are the little problems that sometimes seem to mount 
in geometric progression: 

. The notices being mailed at the wrong time. 

• Not all of the information being complete on the forms so that 
the testing session is delayed and lunch is scheduled only 15 
minutes after the testing session is scheduled to end. 
An error in the computer program which causes some students not 
to get credit for passing the test, etc. 
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All in all, thqre are various categores of problems. Some of j^hich 
have rather straightfoirward solutions. Others are of a more technical 
nature and may require special consideration. Some of the particular 
technical problems faced in our district are discussed in more detail. 

Tne first of several technical problems that required resolution 
was to devise an operational definition of the **8th grade competency'* 
tnat was specified in the adopted board policy. Actually, three forms 
of the operational definition were required: 

. one definition for use with the level 4 form A of the California 
Achievement Tests. This test is administered to all 8th grade 
students in February as part of the regular districtwide testing 
program. It is the first opportunity allowed for students to 
demonstrate that they have achieved competency in reading and in 
math.) 

. a second definition for use with form B of this same test. (This 
test is used as a final examination for high school students who 
are enrolled in special remedial classes) . 

. a third definition for use with the Sequential Tests of Educational 
Progress (STEP). (This test is administered to all high school 
students in the spring of each year as part of the regular 
districtwide testing program.) 

Ti70 different considerations had to be resolved before the operational 
definition for level A of the CAT could be established. 

. The expression '8th grade" in the policy statement implied the 
use of a norm reference standard but the policy did not clearly 
indicate whether this standard should be one of local noms. 
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national norms > urban district norms , or some other set of norms* 
In the case of the Austin policy, however, the discussions by 
the board of trustees prior to adoption of the policy clearly 
Indicated that the national norm referenced standard was Intended 
by the board. This standard was thc^refore incorporated Into the 
operational definition. 
• Since form A of the CAT is administered in February, it would 
seem reasonable to define the cutoff score as the 50th percentile 
point for a midyear testing. There was some discussion however 
of insisting that the "8th grade expectancy'* should refer to com- 
petency at the end of the 8th grade year rather than at the mid- 
point of the year. If this alternative were adopted, the cutoff 
score would have to be the end of year 50th percentile point 
rather the midyear 50th percentile point. Practical considerations 
resolved this issue in favor of the use of the midyear 50th per- 
centile point as the cutoff score. Host students who would score 
an end of year 50th percentile at the end of the 8th grade would 
be expected to score a midyear 50th percentile in the middle of 
the 8th grade year* There seemed to be little purpose in denying 
competency status to these students on the 8th grade administration 
of the CAT since most of these students would achieve competency 
an3rway next year^durlng the next year's testing session. 
The final operational definition of "8th grade competency'' for the 
CAT level 4 form A (the 8th grade test) was thus: 

a score of 50th percentile or higher, based on 8th grade national 
norms for midyear testing. 

The operational definitions for form B of the CAT and for the STEP 
tlien had to be defined so as to be equivalent to the above operational 
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definition. For form B of the CAT, the corresponding operational defi- 
nition was very straightforward - it is identical to the def initiou for 
form A except that the form B norm tables to determine the raw score 
corresponding to the 8th grade midyear 50th percentile point (even though 
this test is used for high school students). 

The^operational definition of "8th grade competency'/ for the STEP 
administrations at high school wa^ not so straightforward. No equivalency 
studies were available to rel^ite the CAT and the STEP and the Austin 
Evaluation Office did not have the resources to conduct such a study. 

Although the Anchor study results were restricted to reading tests 
and for grades 4-6 only, some extrapolation of these Anchor study results 
to the Austin needs appeared to be the most reasonable alternative avail- 
able. This extrapolation was based on the obsexrvation that for each of 
grades 4, 5, and 6, a 50th percentile score on the CAT Reading Test Level 
4 was approximately equivalent to a 45th percentile score on the STEP 
JLevel 3 Reading Test. Assuming that this relationship is also approxi- 
mately true at the 8th grade and is approximately true for math tests 
as well as for reading, the following rationale can be used to determine 
"8th grade competency" for STEP administrations that would be equivalent 
to that used for the CAT: 

. the 8th grade 50th percentile point on the CAT is "equivalent*' 
to the 45th percentile on the STEP, level 3 (based on the extra- 
polations of the Anchor study, discussed above). 
. Using the scale scores that link the different levels of the 

STEP, it is found that the 45th percentile of STEP level 3 corres- 
ponds, on level 2, to a raw score of 

28 for the STEP level 2 Reading Test, 
26 for the STEP level 2 Computation Test, and 
20 for the STEP level 2 Math Concepts Test. 
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The operatiofml definition of **8th grade competency" for the high 
school adminiscracions of the STEP are thus the three raw scores listed 
above • 

The logic that was involved in the development of these STEP com- 
petency scores was rather tenuous and disquieting. Therefore, the Austin 
Evaluation Office staff examined some hit-and-miss contingency tables which 
were created with existing data to ascertain the adequacy of the STEP 
operational definition. For both reading and math, the consistency 
of the ^'pass/no pass" decision between the 8th grade CAT administration 
and the 9th grade STEP administration is reasonably high. The two tables 
below display these results. 



9th Grade 
STEP Reading 



8th Grade 
CAT 

Reading 



9th Grade 
STEP Math 





pass 


fail 


8th Grade 




pass 


fail 


pass 


47% 


11%~' 


CAT 


pass 


40% 


5% 


fail 


5% 


37% 


Math 


fail 


8% 


47% 



The second of the technical problems that required resolution was 
the problem of test security. This problem is still not adequately re- 
solved. A long-range solution to the issue is now under consideration 
and is discussed in this paper. The essence of the problem is that the 
same test is administered a number of times each year. Consequently, 
memory effects may occur, and the possibility also exists for one copy 
of the test being circulated through the underground to students. Since 
the publisher of the test provides no rigid control it is also possible 
that copies of the tes could be obtained by students simply by ordering 
them from the publisher. 
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The only adequate method of maintaining physical security seems to 
involve the creation of ^a new test for each testing situation* Naturally, 
each of these different test forms would. need to be a measure of the same 
variable (competency in reading, or in math). Also, the cutoff score on 
each form would have to be tied back in to the original operational de- 
finition of "8th grade competency'* as identified on the CAT level 4 form A. 
15,16 

The Rasch model appears promising as a solution to these problems."* 

The following tenative plan is being explored, using the reading competency 
as an example: 

1. All items on the CAT Reading Test would be Rasch-calibrated on 
a logit scale. 

2. Assuming that all items satisfied the goodness of -fit test for 
the Rasch model, then the estimate of person ability for a raw 



score that corresponds to the 8th grade midyear 50th percentile 
point would be defined as the cutoff point on the logit scale. 
These test items form the initial kernel of an item pool to be 
created. 

3. Additional test items from other sources are calibrated and are 
linked to the CAT items already in the item pool. 

4. Each test form to be used in an administration would be created 
by drawing items from the pool that were as near as possible to 
the logit scale estimate of parson ability that is the cutoff 
point. 

Regarding point ill in the above, some discussion has insued as to 
the proper course of action if one or more of the CAT items do not satisfy 
the Rasch goodness of fit test. George Ingebo of the Portland Schools 
suggests that if only a few of the CAT test items fail to meet the good- 
ness of fit criteria, this will have only negligible impact on the final 
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results • 16 

A final technical problem (actually a class of loosely related problems) 
Involves the validity of the measurement procedure* 

* The question of test bias is still an unresolved issue. One of 
the difficulties in examining this issue is that by most models 
of test fairness, an external criterion must be identified and 
measured. Such a task would be difficult enough under objective 
procedures. However, the reality is that such a selection would 
also be so intertwined in different political and social factions 
as to be even more complicated. The other great difficulty here 
is that there are many different models of test fairness and for 

the most part they are not internally consistent; that is, a measure- 
ment that is fair under one model is seldom fair under another 
model. 

• A final and perhaps more fundamental problem involves the question 
of whether the board's understanding of what ''8th grade competency** 
means is identical to the operational definition actually used. 
Consider for example the following experiment that might be per- 
formed. 

The cutoff score for the 8th grade CAT Reading administration, 
in terms of raw score units, is 55 raw score (out of a maximum 
of 85 test items). Suppose that the board members were asked 
to examine each of these 85 test items and to reflect on the 
content knowledge that would be involved in correctly answer- 
ing each of these items. After this task was completed, the 
board members would then be asked the following question: 

"Do you agree that any student who can correctly answer 55 
of these items (65%) has an 8th grade competency?** 

A conjecture as to how board members might answer the question 

would be hazardous and will not be offered in this paper. However, 
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considering how educational jargon is bandied about without serious effort 
at communicating the intent of the jargon, it should not be considered 
out of the realm of possibility that this simple experiment might reveal 
a serious gap between the understanding of one or more of the board 
members and those who happily created those wonderfully precise and highly 
measureable operational definitions*^^ 
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